Ground Truth Protocol Guidelines

Tommy Macrina's Directives (August - September 2025)


Executive Summary

This document compiles protocol clarifications and best practices for ground truth segmentation tasks as directed by Tommy Macrina in the #putracers Slack channel between August 1 and September 3, 2025. All guidelines are referenced to specific Slack conversations for verification.


I. Mitochondria Segmentation Protocol

1.1 Individual Segmentation Required

Each mitochondrion must be assigned a separate segment ID and color. Do not paint all mitochondria with the same segment.

Reference: Message m10

1.2 Oversegmentation Directive

The original task description stating "do NOT oversegment mitochondria" was incorrect and should be disregarded. Mitochondria must be oversegmented individually.

Reference: Message m8

1.3 Membrane Handling

Do not paint mitochondrial membranes. Leave a visible border/gap around each mitochondrion to represent the membrane boundary.

References: Message m8, Message m15


II. Myelin Segmentation Protocol

2.1 Inner Boundary (Cell Interface)

No padding between myelin and the cell it ensheathes. The myelin segment should directly contact the inner cell.

Reference: Message m15

2.2 Myelin-to-Myelin Contact

No padding between adjacent myelin segments. Myelin segments may touch each other.

Reference: Message m15

2.3 Outer Boundary (Adjacent Cells)

Myelin should approach but not touch the membranes of neighboring cells. Maintain a small gap between myelin exterior and other cell membranes.

Reference: Message m15


III. Upsampled Image Best Practices

3.1 Two-Pass Labeling Approach

Recommended workflow:

  1. First pass: Paint annotations on the original image (thicker sections)
  2. Second pass: Copy labels onto the upsampled image and connect objects naturally

Reference: Message m2

3.2 Multi-Resolution Viewing

Working exclusively from upsampled images can be misleading. Always cross-reference with original image data.

References: Message m3, Message m4

Visual Reference: Side-by-side comparison

3.3 Handling Artifacts

"Hallucinations" (artificial structures in upsampled data) should be ignored. Focus on anatomically sensible structures.

References: Message m1, Message m11


IV. Ambiguous Cases & Edge Cases

4.1 Uncertain Organelles

When uncertain whether an object is a mitochondrion (vs. peroxisome, lysosome, etc.), segment it as a mitochondrion. Objects can be removed during review if needed.

Reference: Message m9

4.2 EM Damage Regions

Follow team consensus for handling EM damage. Specific protocols to be determined based on damage type.

Reference: Message m17


V. Workflow & Quality Assurance

5.1 Mandatory Peer Review

Before marking tasks as finished, at least one other team member must inspect your work. Prioritize sharing areas where you struggled.

Reference: Message m13

5.2 Sharing Annotations for Review

Correct procedure in WebKnossos:

  1. Navigate to Tasks
  2. Search for tasks under the relevant project
  3. Click the plus symbol (+) for the task
  4. Select "Actions" button (right side)
  5. Choose "View"

References: Message m13, Message m14


VI. WebKnossos Tools & Techniques

6.1 Interpolation Tool

The volume interpolation feature is particularly useful for objects perpendicular to the sectioning plane in higher z-resolution data. May require manual cleanup.

Reference: Message m7

Documentation: WebKnossos Interpolation Guide

6.2 3D Mesh Visualization

Reference: Message m12

Documentation: Keyboard Shortcuts

6.3 SAM2 AI Segmentation (Experimental)

New AI-assisted segmentation tools available on experimental server:

Reference: Message m18


VII. Additional Resources

Key Discussion Threads

External Links


Reference Messages

Below are the complete Slack message threads referenced throughout this document.


Message & Thread Index

m1 — putracers — 2025-08-05 19:38 UTC — Hallucination handling directive

m2 — putracers — 2025-08-05 19:41 UTC — Two-pass labeling approach

m3 — putracers — 2025-08-05 19:41 UTC — Upsampled image caution

m4 — putracers — 2025-08-05 20:18 UTC — Side-by-side comparison link

m5 — putracers — 2025-08-05 20:41 UTC — Volume visibility troubleshooting

m6 — putracers — 2025-08-05 20:41 UTC — Segmentation visibility confirmation

m7 — Thread in putracers — 2025-08-01 13:34 UTC — Interpolation tool discussion

m8 — Thread in putracers — 2025-08-01 15:25 UTC — Mitochondria oversegmentation clarification

m9 — Thread in putracers — 2025-08-04 18:54 UTC — Ambiguous organelle decision

m10 — Thread in putracers — 2025-08-05 14:31 UTC — Individual mitochondria segmentation

m11 — Thread in putracers — 2025-08-05 19:38 UTC — Difficult hallucinations discussion

m12 — Thread in putracers — 2025-08-05 20:03 UTC — 3D meshing usage

m13 — Thread in putracers — 2025-08-05 20:20 UTC — Peer review requirement

m14 — Thread in putracers — 2025-08-06 14:19 UTC — Task sharing troubleshooting

m15 — Thread in putracers — 2025-08-06 15:17 UTC — Comprehensive protocol best practices

m16 — Thread in putracers — 2025-08-14 17:58 UTC — Task transition and IP updates

m17 — Thread in putracers — 2025-08-18 19:37 UTC — EM damage handling

m18 — Thread in putracers — 2025-08-31 19:07 UTC — SAM2 AI tool deployment


Standalone Messages by Tommy Macrina

m1 — Hallucination Handling

Channel: putracers

Date: 2025-08-05 19:38 UTC

Author: Tommy Macrina

View in Slack

From my spot checks, you've all done well: just ignore them.

Context: Response regarding how to handle "hallucinations" in upsampled segmentation data.

m2 — Two-Pass Labeling Strategy

Channel: putracers

Date: 2025-08-05 19:41 UTC

Author: Tommy Macrina

View in Slack

I realize now that we should probably handle labeling this data in two passes:

1. paint the original image (thicker sections)

2. copy those labels onto the upsampled image (computationally thinner sections),

   and just connect the objects as "naturally" as possible

m3 — Upsampled Image Warning

Channel: putracers

Date: 2025-08-05 19:41 UTC

Author: Tommy Macrina

View in Slack

I've found only working from the upsampled image to be misleading.

m4 — Comparison Resource

Channel: putracers

Date: 2025-08-05 20:18 UTC

Author: Tommy Macrina

View in Slack

Here's a side-by-side of the original image and the upsampled image:

https://spelunker.cave-explorer.org/#!middleauth+https://global.daf-apis.com/nglstate/api/v1/5712377138905088

m5 — Troubleshooting Visibility

Channel: putracers

Date: 2025-08-05 20:41 UTC

Author: Tommy Macrina

View in Slack

Open left sidebar and see if "volume" is flipped on?

m6 — Segmentation Confirmation

Channel: putracers

Date: 2025-08-05 20:41 UTC

Author: Tommy Macrina

View in Slack

I see your segmentation still.


Threaded Discussions with Tommy Macrina

m7 — Interpolation Tool Usage

Channel: putracers

Date: 2025-08-01 13:34 UTC

View in Slack

Thread Summary: Discussion of WebKnossos interpolation tool effectiveness

Ryan (2025-08-01 13:34:20 UTC):

Oh that's what that does. That would be useful with the larger, simpler segments

we typically start off with, but the smaller neurites need more guidance. I agree

with Jay that it might require clean up but I'll give it a try

Tommy Macrina (2025-08-01 15:30:16 UTC):

Yeah, this has been our general experience, too.

Tommy Macrina (2025-08-01 15:31:38 UTC):

Because this image has a higher z-resolution, more objects that are perpendicular

to the sectioning plane don't change that much. So you might find more objects

where interpolation is useful.

Ryan (2025-08-01 15:59:35 UTC):

Yea I tried it on a simple round segment with minimal movement between slices with

a 2 slice gap and it was pretty perfect. But I tried 5-6 slices in between and it

got a bit confused by the size change in the segment and under-colored. But I'm

gonna keep playing with it and find the sweet spot

m8 — Mitochondria Oversegmentation Clarification

Channel: putracers

Date: 2025-08-01 15:25 UTC

View in Slack

Thread Summary: Critical correction regarding mitochondria segmentation protocol

Jay (2025-08-01 15:25:29 UTC):

@Tommy Macrina @Eric Are we definitely oversegmenting mitochondria?

The task directions explicitly say not to.

Attached: image.png

Tommy Macrina (2025-08-01 15:26:57 UTC):

Yes, it was a mistake to assign that task description.

Jay (2025-08-01 15:27:16 UTC):

Ok, thanks for the confirmation, we'll oversegment!

Eric (2025-08-01 15:27:41 UTC):

Sorry for the confusion, just was getting to this

bun (2025-08-01 15:27:46 UTC):

do we leave a border between mitos?

bun (2025-08-01 15:27:57 UTC):

for membrane

Tommy Macrina (2025-08-01 15:28:20 UTC):

Yes, try not to paint the mitochondria's membrane.

bun (2025-08-01 15:30:37 UTC):

is there a way we can get Kisuk's tool that expands or reduces the segments

so we can make the membrane unpainted while still using the fill tools?

Tommy Macrina (2025-08-05 19:43:23 UTC):

> Kisuk's tool that expands or reduces the segments

This is some post-processing he does. Handling it manually for now is the

right call, until we have more experience with this data.

Tommy Macrina (2025-08-05 19:43:50 UTC):

It'd be nice if webknossos made it easy to extend their toolkit...

[Thread continues with discussion of shape deforming tools and interpolation - see full message archive for complete context]

m9 — Ambiguous Organelle Segmentation

Channel: putracers

Date: 2025-08-04 18:54 UTC

View in Slack

Thread Summary: Guidance on handling uncertain organelle identification

bun (2025-08-04 18:54:04 UTC):

should i over segment this object in the cell body? i'm not 100% sure its a

mitochondria. it might be a peroxisome or lysosome or something else

@Tommy Macrina @Eric

Attached: image.png

Tommy Macrina (2025-08-05 19:33:04 UTC):

Link or location?

Tommy Macrina (2025-08-05 19:34:15 UTC):

It's not jumping out at me in your volume. I think your volume looks pretty good.

bun (2025-08-05 19:34:48 UTC):

295127, 258550, 3573

bun (2025-08-05 19:35:24 UTC):

i went ahead and segmented it. figured we could always get rid of it if we need to.

Tommy Macrina (2025-08-05 19:35:29 UTC):

Yeah, I think you make the right call to segment it.

m10 — Individual Mitochondria Segmentation

Channel: putracers

Date: 2025-08-05 14:31 UTC

View in Slack

Thread Summary: Clarification on segment ID assignment for mitochondria

Jay (2025-08-05 14:31:09 UTC):

@Eric @Tommy Macrina Point of clarification: should all mitochondria be segmented

with the same segment/color, or should they each be given an individual segment/color?

Sorry, it's been so many years since I've done one of these tasks that I don't

remember what the correct protocol is

Tommy Macrina (2025-08-05 19:27:41 UTC):

Different segment for each mitochondria.

m11 — Difficult Hallucinations

Channel: putracers

Date: 2025-08-05 19:38 UTC

View in Slack

Thread Summary: Discussion of challenging artifacts in upsampled data

Tommy Macrina (2025-08-05 19:38:27 UTC):

Does anyone have a "hallucination" that they found particularly difficult?

bun (2025-08-05 19:52:18 UTC):

294885, 258672, 3576,

294908, 258553, 3589,

295009, 258619, 3565

bun (2025-08-05 19:52:41 UTC):

just off the top of my head. these areas were a bit tricky

Ryan (2025-08-06 15:09:27 UTC):

So far, because these slices are artificial, I've just done my best to annotate

something that makes sense with the flow/movement and what makes sense with known

morphology, even if it slightly goes against the shape/borders the up-sampling

came up with

m12 — 3D Mesh Viewing

Channel: putracers

Date: 2025-08-05 20:03 UTC

View in Slack

Thread Summary: Introduction to 3D visualization features in WebKnossos

Tommy Macrina (2025-08-05 20:03:05 UTC):

How much have you all used the ad hoc meshing in webknossos?

Jay (2025-08-05 20:14:40 UTC):

The interpolation? I tried it out for a while, but I found it produced enough

errors that I was spending just as much time cleaning it up as I would be

tracing the slices myself so I stopped using it

Tommy Macrina (2025-08-05 20:17:36 UTC):

No. If you right-click on a segment, you can generate a 3D mesh.

Jay (2025-08-05 20:24:52 UTC):

I didn't even realize there was a 3D view. How do you access it? I don't see

any menu options, and the "keyboard shortcuts" help option is just a 404 error

Tommy Macrina (2025-08-05 20:46:19 UTC):

Hit "." to toggle between all views and the view you hover over.

Tommy Macrina (2025-08-05 20:46:28 UTC):

Docs for webknossos are online, too

Tommy Macrina (2025-08-05 20:46:29 UTC):

https://docs.webknossos.org/webknossos/ui/keyboard_shortcuts.html

m13 — Peer Review Requirement

Channel: putracers

Date: 2025-08-05 20:20 UTC

View in Slack

Thread Summary: Mandatory quality control process

Tommy Macrina (2025-08-05 20:20:14 UTC):

Before calling these tasks finished, would you mind having at least one other

team member inspect your work? You might share the places where you struggled

the most.

bun (2025-08-05 20:39:13 UTC):

can you remind us how to do that in webknosses. i can't seem to "View" other

peoples tasks

Tommy Macrina (2025-08-05 20:45:02 UTC):

Go to tasks, search for tasks under project `mec_cutout_0_upsampled`, click on

plus symbol for any task, go to `actions` button on the right side, select `View`.

Attached: image.png

bun (2025-08-05 20:48:52 UTC):

thanks!

m14 — Task Sharing Troubleshooting

Channel: putracers

Date: 2025-08-06 14:19 UTC

View in Slack

Thread Summary: Technical issues with viewing shared annotations

[Extended thread discussing WebKnossos UI challenges and annotation sharing - abbreviated for brevity]

Tommy Macrina (2025-08-06 15:22:36 UTC):

No idea how that link doesn't want to load the image data.

Tommy Macrina (2025-08-06 15:22:57 UTC):

If you work through tasks > actions > view, it shows up:

http://wk.zetta.ai:9000/annotations/688a4519010000db0079097b#295030,259728,3539,0,0.35

Tommy Macrina (2025-08-06 15:40:04 UTC):

Hit `.`

Tommy Macrina (2025-08-06 15:40:59 UTC):

Ohhh... maybe Austin copied the task to his own account. That could explain

the missing image.

m15 — Comprehensive Protocol Best Practices

Channel: putracers

Date: 2025-08-06 15:17 UTC

View in Slack

Thread Summary: Comprehensive discussion of segmentation best practices

bun (2025-08-06 15:17:12 UTC):

As for the GT best practices I know Tommy mentioned the cutouts shouldn't have

had that description applied: "Segment all cells. Segment myelin as a separate

object, without padding between the myelin and neighboring objects. Pay special

attention to segment any invaginations, or invagination-like structures. Do NOT

oversegment mitochondria. Leave extracellular space unsegmented."

I think the most up to date ground truth best practice would be this: "Segment

all cells. Segment myelin as a separate object, *without padding between* the

myelin and neighboring objects. Pay special attention to segment any invaginations,

or invagination-like structures. Segment mitochondria *without membrane painted*.

Segment extracellular space."

@Tommy Macrina Is that fair to say?

Ryan (2025-08-06 15:29:49 UTC):

I think we're supposed to avoid the mitochondria's border/membrane:

[link to earlier message]

Jay (2025-08-06 15:30:26 UTC):

Yeah to clarify, I think the following schema should be correct for myelin:

1. No padding between myelin and the cell it surrounds

2. No padding between myelin and other myelin

3. Myelin should butt up against the cell membranes of other cells, but not

   touch the actual segment

Attached: image.png

bun (2025-08-06 15:35:06 UTC):

Yes I think this is mostly correct. The outside of myelin shouldn't touch other

cells/membranes, but it might be the case that it can touch other myelin. Not

100% about that though. The inside should touch the cell it is myelinating like

shown. @Tommy Macrina

bun (2025-08-06 15:35:41 UTC):

@Ryan yes, "Segment mitochondria *without membrane painted*."

Ryan (2025-08-06 15:36:34 UTC):

Oh yes I definitely misread that!

Tommy Macrina (2025-08-06 15:39:49 UTC):

Yeah, that's right, @bun

m16 — Task Transitions and IP Updates

Channel: putracers

Date: 2025-08-14 17:58 UTC

View in Slack

Thread Summary: Project management and technical access

Jay (2025-08-14 17:58:06 UTC):

@Austin.Burke @bun @twister @wileykit @Ryan This round of GTs are up; if y'all haven't been invited to the project yet LMK & I'll get that taken care of

[Thread primarily regarding task scheduling and IP whitelist updates]

Tommy Macrina (2025-08-14 21:30:22 UTC):

@bun yeah, if your IP address changes, it needs to be updated.

m17 — EM Damage Protocol

Channel: putracers

     Ben Silverman (2025-08-18 19:37 UTC):

I think what we've been doing is if it's inside the segment you label it as the segment it's in. if it's on the border of the segment i think we do as shown in this pic iirc.

View in Slack

Thread Summary: Handling electron microscopy damage in segmentation

Austin.Burke (2025-08-18 19:37:34 UTC):

@Tommy Macrina What would be the protocol for EM damage like this? Should it be

left alone as is or segmented as the same segID or a separate one?

Attached: image.png

Tommy Macrina (2025-08-20 13:16:58 UTC):

Agree with @bun below

[Note: bun's recommendation not included in Tommy's message; refer to full thread for context]

m18 — SAM2 AI Tool Deployment

Channel: putracers

Date: 2025-08-31 19:07 UTC

View in Slack

Thread Summary: New AI-assisted segmentation capabilities

ranl (2025-08-31 19:07:47 UTC):

@channel, I deployed a experimental webknossos server at http://ranl.zetta.ai:9000/,

upgraded to the new 25.09 release and include new features like SAM2. The new

server does not contain any old data, so you have to register again. Migrate the

database might be difficult so we probably switch after the current tracing tasks

are finished, meanwhile you can give it a try and report if there are any issues.

Tommy Macrina (2025-09-02 10:38:04 UTC):

@ranl Can you give me access?

ranl (2025-09-02 12:49:03 UTC):

done

Tommy Macrina (2025-09-02 13:56:33 UTC):

Thanks!

Tommy Macrina (2025-09-02 14:06:09 UTC):

Pretty good at bigger convex cross sections without many intercellular objects. Cool!

Tommy Macrina (2025-09-02 14:06:29 UTC):

Got it working on some objects for 20 sections just pointing & clicking.

[Thread continues with technical discussion of SAM2 parameters and performance - see full archive for complete details]


Document Information

Source: Slack workspace (#putracers channel)
Date Range: August 1 - September 3, 2025
Primary Subject Matter Expert: Tommy Macrina
Compilation Date: November 6, 2025
Total Messages Referenced: 18 (6 standalone + 12 threaded discussions)

Images: Referenced images are stored in Ground Truth Protocol: Tommy Macrina Reference Set directory on google drive

This document was automatically generated from Slack message exports and formatted for professional distribution.


Document Control

Created: November 6, 2025
Last Updated: November 6, 2025
Owner: Benjamin Silverman
Review Frequency: As needed when processes change